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(54) Image processing apparatus for displaying three-dimensional image 

(57) A stereoscopic image processing apparatus 
which can automatically assign stereoscopic viewing 
parameters to VRML data in which no stereoscopic 
viewing parameters are defined. VRML data is input, 
and the user sets or changes stereoscopic viewing 
parameters (initial values of the base line length and 
convergence angle) required for stereoscopic viewing 
on the basis of the input VRML data. The initial value of 
the base line length is set based on the maximum size 
of the range of the space of the VRML data, and the ini- 
tial value of the convergence angle is set at "0" .These 
initial values are arbitrarily changed at a user interface 
(103). 




^ — 1-4 



CO 



Q. 
LU 



Primed by Xerox (UK) Business Services 

2.16.7/3.6 



BNSDOCID: <EP 096941 8A2J_> 



1 



EP0 969 418 A2 



2 



Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to an image 
processing apparatus for converting space data 
described in a language or format that processes a 
three-dimensional virtual space such as a VRML (Vir- 
tual Reality Modeling Language) or the like, into image 
data that can be stereoscopically observed, and to a 
viewer apparatus for displaying such image data. 
[0002] The present invention also relates to a user 
interface apparatus for changing stereoscopic viewing 
parameters, a stereoscopic display buffer control 
method and apparatus for obtaining stereoscopic view- 
ing, and a program storage medium for storing their pro- 
grams. 

[0003] As the performance of computers has 
improved in recent years, three-dimensional computer 
graphics (CG) has increasingly prevailed. In addition, 
along with rapid spread of WWWs (World Wide Webs) 
on the Internet, a VRML as a language that processes a 
three-dimensional space on the WWW has been stand- 
ardized, and has been prevalently used. 
[0004] The VRML 2.0 as the currently available latest 
version is used for building a virtual mall for electric 
commerce, managing three-dimensional data of. e.g., 
CAD in an intranet, and so forth. 
[0005] On the other hand, the assignee of the present 
invention has developed a rear-cross lenticular 3D dis- 
play (to be referred to as a stereoscopic display appara- 
tus hereinafter to be distinguished from a two- 
dimensional display apparatus) as a stereoscopic dis- 
play apparatus that allows the observer to directly per- 
ceive depth. In addition to this display apparatus, many 
stereoscopic display apparatuses have been proposed. 
[0006] In order to display a three-dimensional space 
described in the VRML 2.0, a VRML viewer apparatus is 
required. However, a normally prevalent VRML viewer 
apparatus can only make two-dimensional display (to 
be referred to as a "two-dimensional display apparatus 
hereinafter in contrast to the "stereoscopic display 
apparatus") as final display using a normal monitor, 
although a three-dimensional space is described. 
[0007] This is because some parameters (to be 
referred to as "stereoscopic viewing parameters" here- 
inafter) for attaining stereoscopic viewing are not sup- 
ported by the VRML 2.0. 

[0008] In other words, there is no conventional stere- 
oscopic display apparatus that can efficiently change 
the "stereoscopic viewing parameters" in utilizing space 
data described in the language or format such as VRML 
that cannot describe the stereoscopic viewing parame- 
ters. This is the first problem the conventional stereo- 
scopic display apparatus suffers. 
[0009] In order to realize stereoscopic viewing on a 
display, for example, in, e.g., the rear-cross lenticutar 3D 
display proposed by the assignee, right and left images 



must be alternately written in one frame buffer stripe by 
stripe. 

[001 0] The problem in the conventional buffer control 
method will be explained below using the flow chart in 
5 Fig. 18. 

[0011] Conventionally, as shown in the control 
sequence in Fig. 18, an image to be seen when the 
three-dimensional space is viewed by the right eye is 
generated, and three-dimensional space data is ren- 
te? dered on the basis of the viewpoint position and gaze 
direction of the right eye in step S50. With this process, 
a right eye image as a rendering result is stored in frame 
buffer A. 

[0012] In step S52, an image to be seen when the 
15 three-dimensional space is viewed by the left eye is 
generated, and three-dimensional space data is ren- 
dered on the basis of the viewpoint position and gaze 
. direction of the left eye. As a result, a left eye image as 
a rendering result is stored in frame buffer B. 
20 [0013] In step S54, frame buffer C is prepared, and 
data required for right eye display is read out from the 
contents of frame buffer A and is written in frame buffer 
C. 

[0014] In step S56, data required for left eye display is 
25 read out from the contents of frame buffer B and is writ- 
ten in frame buffer C. In this manner, the buffer control 
method shown in Fig. 18 requires three buffers. 
[0015] Fig. 19 shows a buffer write control sequence 
according to another prior art. The buffer control method 
30 shown in Fig. 1 9 requires two buffers. 

[001 6] In this fashion, both the methods shown in Figs. 
18 and 19 require two or more frame buffers, and are 
disadvantageous in cost. 

[0017] Also, inter-buffer data transfer that requires 
35 processing for reading out data from a frame buffer in 
which the data has been written, and writing the data in 
another frame buffer is time-consuming processing, and 
lowers the processing speed. 

40 SUMMARY OF THE INVENTION 

[0018] The present invention has been made in con- 
sideration of the conventional problems, and has as its 
object to provide a stereoscopic image processing 

45 apparatus, user interface apparatus, and stereoscopic 
image display apparatus, which can assign parameters 
to three-dimensional space data in which stereoscopic 
viewing parameters are not defined. 
[0019] ft is another object of the present invention to 

so provide a user interface apparatus which can freely 
change the stereoscopic viewing parameters. 
[0020] It is still another object of the present invention 
to provide a buffer control apparatus and method, which 
requires only one frame buffer, and can obviate the 

55 need for reading out any data from that frame buffer. 
[0021 ] In order to achieve the above object, k stereo- 
scopic image processing apparatus according to the 
present invention comprises: 
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means for receiving and storing three-dimensional 
space data; and 

means for generating a parameter required for ster- 
eoscopic viewing on the basis of the received three- 
dimensional space data. 5 

[0022] According to a preferred aspect of the present 
invention, the parameter defines a base line length of a 
user. Also, according to a preferred aspect of the 
present invention, the parameter defines a convergence 10 
angle of a user. The base line length and convergence 
angle are mandatory parameters required to realize 
stereoscopic viewing. 

[0023] Since the user often does not know that an 
input three-dimensional space data requires parameter is 
setup, it is preferable to automatically set parameters. 
Hence, according to a preferred aspect of the present 
invention, the apparatus further comprises means for 
determining necessity of generation of the parameter on 
the basis of a file extension of the received three-dimen- 20 
sional space data or attribute information appended the 
data file. 

[0024] According to the present invention, a user inter- 
face apparatus used upon displaying a stereoscopic 
image, comprises: 25 

user setting means for allowing a user to set a value 
of a stereoscopic viewing parameter; and 
means for changing a value of the stereoscopic 
viewing parameter in accordance with the set value. 30 

[0025] With this user interface apparatus, the param- 
eters can be freely changed to match image data with 
the purpose of the user's application. 
[0026] The user interface is preferably displayed on 35 
the display screen. According to a preferred aspect of 
the present invention, the apparatus further comprises 
display means, and the user setting means displays a 
GUI on the display means. 

[0027] According to a preferred aspect of the present 40 
invention, the GUI is implemented by a slide bar, jog dial 
or wheel. 

[0028] A buffer control method for effectively using a 
buffer according to the present invention is directed to a 
buffer control method upon obtaining stereoscopic view- 45 
ing by displaying right and left eye images written in one . 
frame buffer, comprising the steps of: 

preparing mask information that defines a mask 
function in advance; so 
writing one of the right and left eye images in the 
frame buffer; and 

overwriting the other image on the frame buffer with 
reference to the mask information. Upon overwrite, 
a portion of one image is left and a portion of trje ss 
other image is overwritten on the frame buffer in 
accordance with the mask function of the mask 
information. 



[0029] The above objects are also achieved by a 
buffer control apparatus which has one frame buffer and 
obtains stereoscopic viewing by displaying right and left 
eye images written in the frame buffer, comprising: 

a mask memory written with mask information that 
defines a mask function in advance; and 
control means for writing one of the right and left 
eye images in the frame buffer, and then overwriting 
the other image on the frame buffer with reference 
to the mask information in the mask memory. 

[0030] In this case, according to a preferred aspect of 
the present invention, the mask memory comprises a 
stencil buffer. 

[0031] In the control method according to a preferred 
aspect of the present invention, the mask function is 
determined based on an order of stripes on a display 
apparatus, and a write order of images in the frame 
buffer. 

[0032] In the control method according to a preferred 
aspect of the present invention, the mask information is 
written in a stencil buffer at least before the other image 
is written in the frame memory. 

[0033] Other features and advantages of the present 
invention will be apparent from the following description 
taken in conjunction with the accompanying drawings, 
in which like reference characters designate the same 
or similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0034] 

Fig. 1 is a view for explaining a WWW network to 
which viewer apparatuses according to an embodi- 
ment are connected; 

Fig. 2 is a block diagram showing the arrangement 
of a viewer apparatus 1 000 according to an embod- 
iment; 

Fig. 3 is a perspective view for explaining the 
arrangement of a rear-cross lenticular spectacle or 
3D display 10 used in the viewer apparatus 1000; 
Fig. 4 is an enlarged view of an LCD portion of the 
display 10; 

Fig. 5 is a view for explaining the principle of gener- 
ation of right and left eye images for obtaining ster- 
eoscopic viewing; 

Fig. 6 is a view for explaining the relationship 
between the convergence angle and base line 
length; 

Fig. 7 is a view for explaining the relationship 
between the convergence angle and base line 
length by another description method; 
Fig. 8 is a view for explaining the principle of pro- 
jecting an arbitrary three-dimensional position onto 
the imaging plane of a virtual camera; 
Fig. 9 is a view for explaining the necessity of 
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appropriately setting the base line length; 
Fig. 10 is a view for explaining the necessity of 
appropriately setting the base line length; 
Fig. 11 is a block diagram for explaining the func- 
tional arrangement in the viewer apparatus 1000; 
Fig. 1 2 is a flow chart showing the control sequence 
of a viewer application program; 
Fig. 13 is a view for explaining the principle of set- 
ting an initial value of the base line length; 
Fig. 14 shows a display example of a virtual space 
when a small base line length is set; 
Fig. 15 shows a display example of a virtual space 
when a large base line length is set; 
Fig. 16 is a view for explaining graphical user inter- 
faces displayed on a display apparatus of an 
embodiment; 

Fig. 17 is a table for explaining the order of data 
stored in a frame buffer of in the viewer apparatus 
of an embodiment; 

Fig. 18 is a flow chart for explaining the control 
sequence of frame buffers according to one con- 
ventional method; 

Fig. 19 is a flow chart for explaining the control 
sequence of frame buffers according to another 
conventional method; 

Fig. 20 is a view for explaining an example of mask 
information stored in a stencil buffer 12; 
Fig. 21 is a flow chart for explaining the control 
sequence of frame buffer control according to an 
embodiment; and 

Fig. 22 is a view for explaining the control sequence 
of frame buffer control according to an embodiment. 

DETAILED DESCRIPTION OF THE INVENTION 

[0035] An embodiment in which the present invention 
is applied to a stereoscopic display apparatus (to be 
referred to as a "VRML viewer apparatus" hereinafter) 
for receiving and stereoscopically displaying three- 
dimensional space data described with the VRML 2.0 
using a WWW browser will be described below. This 
stereoscopic display apparatus uses as a display a rear- 
cross lenticular 3D display developed by the present 
applicant so as to give a stereoscopic sense to the user. 
However, as can be seen from the following description, 
the present invention cannot be applied to only virtual 
space data in the VRML 2.0, only virtual space data 
received by a WWW browser, or only the rear-cross len- 
ticular 3D display. 

< System Arrangement > 

[0036] Fig. 1 shows a WWW network to which a plu- 
rality of VRML viewer apparatuses that adopt the 
method of the present invention are connected. The fea- 
tures of the VRML viewer apparatus of this embodiment 
are: 



i: the apparatus has a function of assigning a base 
line length and convergence angle to VRML data 
which is downloaded or transferred from a server 
(1-1, 1-2. 1-3, 1-4, 1-5) at a remote WWW site to a 

5 terminal system 2; 

ii: the user of this viewer apparatus can change the 
values of the base line length and convergence 
angle in VRML data to which initial values of the 
base line length and convergence angle have been 

io assigned; and 

iii: a stereoscopic image can be rendered at high 
speed by effectively using a stencil buffer. 

[0037] Fig. 2 shows the hardware arrangement of a 
is viewer apparatus 1000 of this embodiment. The viewer 
apparatus 1000 has a display 10, a frame buffer 11 for 
temporarily storing data to be displayed on the display 
10, a stencil buffer 12 for storing mask information, a 
CPU 13, a keyboard 14, a mouse 15, and a storage 
20 device 16 like in a normal workstation or personal com- 
puters. 

[0038] The display 1 0 uses a rear-cross lenticular eye- 
glass-less (i.e., naked eye) 3D display proposed by the 
present application to attain stereoscopic viewing. 

25 [0039] As shown in Fig. 3, the rear-cross lenticular 
eye-glass-free 3D display 10 is characterized by insert- 
ing two lenticular lenses (an H lenticular lens 21 and V 
lenticular lens 22) and a checkerboard-like mask panel 
23 between a TFT liquid crystal (LC) panel 20 and back- 

30 light panel 24. The V lenticular lens 22 has a role of 
splitting illumination light coming from the backlight 
panel 24 in the right and left eye directions by changing 
its directionality in units of scanning lines. The H lenticu- 
lar lens 21 prevents crosstalk by focusing light coming 

35 from an aperture of one horizontal line of the checker- 
board-like mask panel 23 onto one scanning line of the 
LC panel 20, and broadens the stereoscopic viewing 
range in the vertical direction by converting light coming 
from the aperture portion into divergent light toward the 

40 observer. 

[0040] Fig. 4 is an enlarged view of a partial region A 
of the LC panel 20 shown in Fig. 3. As shown in Fig. 4, 
an image obtained by synthesizing pairs of right and left 
stereo images for stereoscopic viewing, which are aiter- 

45 nately arranged in horizontal stripe patterns, is dis- 
played on the LC panel 20. For this reason, the following 
merits are expected: no color separation of the liquid 
crystal corresponding to vertical stripes takes place as 
compared to a normal lenticular scheme that receives 

so vertical stripe images, the display 10 is compatible with 
a horizontal stripe synthesized image of a time-division 
field sequential scheme normally used in liquid crystal 
shutter spectacles or the like, and so forth. 

55 ( Rendering for Stereoscopic Viewing > 

[0041] Images of a virtual space constructed using 
three-dimensional geometric data such as VRML data 
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or the like viewed from two viewpoints corresponding to 
the right and left eyes can be rendered using a conven- 
tional CG technique while considering these two view- 
points as virtual cameras, as shown in Fig. 5. In this 
case, data that pertain to the viewpoint positions (R, L), 
gaze direction (vector e), base line length b, and conver- 
gence angle e of the right and left eyes of the user are 
required. 

[0042] Assume that the viewpoint positions (R, L), 
gaze direction (line of sight vector e), base line length b, 
and convergence angle 6 are given. Fig. 6 shows the 
relationship among the viewpoint positions (R, L), gaze 
direction (line of sight vector e), base line length b, and 
convergence angle e when a description method that 
defines the right eye of a dextrocular user as the base 
line position is used. More specifically, if the camera 
coordinate system is represented by (x, y, z), its x-axis 
agrees with a direction from the left eye to the right eye, 
the z-axis agrees with the depth direction, and the y- 
axis agrees with the gaze direction e, the viewpoint 
positions R and L of the right and left eyes are respec- 
tively given by: 

R(xR,y R ,z R ) = (o, o, o) 

L(x L .y L ,z L ) = (x R -b. y R . z R ) 

If (I, m. n) represents the gaze direction vector e R of the 
right eye, the gaze direction vector e L of the right eye is 
also expressed by (I, m, n). According to the description 
of Fig. 6, the vector that expresses the convergence 
angle can be expressed as a vector which has rotated 
an angle e from a line of sight vector e L of the left eye. 
[0043] Fig. 7 explains the description method of the 
base line length b and convergence angle 6 when the 
center of the base line is set at the center between the 
right and left eyes. In this case, the coordinate origin is 
set at the center between the right and left eyes, the x- 
axis agrees with the direction from the left eye to the 
right eye, the y-axis agrees with the vertical direction, 
and the z-axis agrees with the depth direction. Then, the 
right eye is located at a position offset by b/2 to the right, 
and the left eye is located at a position offset by b/2 to 
the left. If 0 represents the convergence angle, the right 
line of sight agrees with a direction that has rotated e/2 
counterclockwise from the line of sight vector e R . and 
the left line of sight agrees with a direction that has 
rotated 0/2 clockwise from the line of sight vector e L . 
[0044] When the viewpoint positions (R, L), gaze 
direction (line of sight vector e), base length b, and con- 
vergence angle e are given, in order to generate right 
and left images using two virtual cameras, as shown in 
Fig. 5, a projection M of a virtual camera placed at the 
position of an eye (right or left eye) onto a two-dimen- 
sional imaging plane (x, y) is obtained. If (Xc, Y c , Zc) 
represents the position of the virtual camera, the projec- 
tion M is expressed by a matrix determined by the view- 
point position (X c , Y c , Z c ), the line of sight direction e, 
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and the field angle, and perspective transformation can 
be normally used. 

[0045] Three-dimensional space data such as VRML 
data or the like contain data that describes a shape 
(three-dimensional position data associated with point 
groups and patch links among points). If P(X, Y, Z) rep- 
resents the position of a given point P in the three- 
dimensional space, a projection of that point P(X, Y, Z) 
onto the imaging plane of the virtual camera is normally 
expressed by the following homogeneous coordinate 
transformation: 



0) 



where M is a 4 x 4 matrix. Note that (x M , y M ) represents 
a two-dimensional coordinate position on the imaging 
plane, and z M corresponds to the depth after perspec- 
tive transformation and can be used upon determining 
the object depth ordering in the depth direction. If equa- 
tion (1) is calculated for all point groups, an image for 
the right or left eye is obtained. If one image (e.g.. right 
eye image) is obtained, the matrix is changed to that for 
the other eye (e.g., (Xc. Y c . Z c ) = (X R , Y R , Z R ) and the 
line of sight direction ept), and equation (1) is calculated 
again. 

[0046] The scheme-for generating the right and left 
eye images for stereoscopic viewing has been 
explained. This scheme itself is known to those who are 
skilled in the art. Also, a scheme that does not use any 
matrix calculation is available. However, in the stereo- 
scopic image display apparatus of the present inven- 
tion, the rendering scheme itself is not limited to any 
specific one. 



40 (VRML Data) 

[0047] Of VRML data, a portion that describes a 
space is composed of: 



data that describes a shape (three-dimensional 
position data associated with point groups); 
data that describes the surface characteristics 
(color, material and texture); and 
data that describes the light source position like in 
other three-dimensional space data. VRML data 
can also describe initial values, several sets of 
viewpoint positions and gaze directions, a method 
of operating a dynamically moving object, a method 
of generating sound, a script language that can 
execute a procedure, such that sound is generated 
if a given object is touched. Of these contents, 
those except for the viewpoint positions and gaze 
directions are not directly related to the present 
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invention, and a detailed description thereof will be 
omitted. 

[0048] However, in data defined by VRML, the "view- 
point position" and "gaze direction" of the user can be 5 
described, but neither data of the "base line length" nor 
"convergence angle" are assigned. It is hot always nec- 
essary to describe "viewpoint position" and "gaze direc- 
tion". This is because the "viewpoint position" and "gaze 
direction" need not be assigned at the time of genera- io 
tion of VRML data since they are used at the time of ren- 
dering a virtual space in practice so as to allow the user 
to freely move his or her viewpoint in the virtual space. 
The same applies even when VRML data is to be two- 
dimensionally or stereoscopically displayed and, hence, is 
the "viewpoint position" and "gaze direction" of the user 
are not mandatory for VRML data. 
[0049] Upon rendering VRML data, the "base line 
length" and "convergence angle" of the user can be 
arbitrary to a certain extent. The size of a space 20 
expressed by VRML data need not be expressed with 
reference to the size of a human, but can be freely set 
according to the will of the data creator. Hence, in the 
VRML data, it is nonsense to define the "base line 
length" and "convergence angle" with respect to actual 25 
size of the person who wants to experience the virtual 
space. For example, when a molecular structure model 
is rendered in the virtual space, the size of a molecule is 
very different from the base line length, and it is essen- 
tially nonsense to define the "base line length" in 30 
accordance with the size of a humanr 
[0050] The present inventor found that the extent that 
the "base line length" and "convergence angle" of the 
user can be arbitrary is not so greater than the "view- 
point position" and "gaze direction". That is, the "view- 35 
point position" and "gaze direction" of the user are 
mandatory when VRML data is displayed on a two- 
dimensional image or a stereoscopic viewing image is 
to be rendered. However, the "base line length" and 
"convergence angle" are not required when only one 40 
two-dimensional image is generated. On the other 
hand, in order to present stereoscopic viewing to the 
user, the "base line length" and "convergence angle" 
are required. For example, upon zooming from a macro 
world to a micro world (e.g., upon rendering a scene the 45 
observer who is looking into a small hole in a broad 
room enters that small hole), when a two-dimensional 
image is rendered, the "base line length" and "conver- 
gence angle" are not in question, but the moving speed 
alone is in question. so 
[0051] On the other hand, assume that in a room 50 
with a small home 40, as shown in Fig. 9, the observer 
approaches the "hole" from a position separated from 
that hole, and enters the hole, while experiencing three- 
dimensional stereoscopic display. When the viewpoint 55 
position is considerably separated from the hole 40, the 
right and left eye images have no differences, and no 
problem is posed. However, when the observer comes 



close to the hole 40, if the base line length (the distance 
between the right and left eyes of a person is 6.5 cm) is 
larger than the diameter (5 cm in the example in Fig. 10) 
of that hole, as shown in Fig. 10, and the user wants to 
look into that hole, stereoscopic viewing rendering loses 
its meaning. That is, in the example in Fig. 10, since the 
base line length is larger than the diameter of that hole, 
the left eye cannot look into the hole 40. Two-dimen- 
sional image rendering does not suffer such problem. In 
case of the example shown in Fig. 10, if a user interface 
that allows the user to shorten the base line length is 
given, a stereoscopic image as if the user were entering 
the hole (that is, the user himself or herself were minia- 
turized) and were looking the interior of the hole can be 
rendered. 

(Automatic Assignment of Base Line Length and Con- 
vergence Angle > 

[0052] The viewer apparatus of this embodiment auto- 
matically assigns a base line length and convergence 
angle to externally received VRML data without the 
intervention of the user, in consideration of the fact that 
the VRML data is not assigned any base line length and 
convergence angle. 

[0053] Fig. 1 1 explains the data flow when the viewer 
apparatus shown in Fig. 2 acquires VRML data from a 
WWW site via a WWW browser, assigns a base line 
length and convergence angle to that VRML data, gen- 
erates right and left eye images 105 and 104 at an arbi- 
trary viewpoint position, and displays the generated 
images on the stereoscopic display 10. 
[0054] More specifically, referring to Fig. 1 1 , a viewer 
application program 1 00 is a helper application program 
of a WWW browser 1 01 that runs under the control of an 
operating system (OS) 102. The viewer apparatus 1000 
of this embodiment has the storage device 1 6 for storing 
the processing sequence (Fig. 12) of the viewer applica- 
tion program 100, virtual space data (three-dimensional 
space data), and processing contents (space operation 
parameters and stereoscopic viewing parameters), as 
shown in Fig. 2. The virtual space data (i.e., three- 
dimensional space data) shown in Rg. 2 corresponds to 
VRML data shown in Fig. 1 1 . Also, the space operation 
data shown in Fig. 2 includes data which are required 
upon making stereoscopic viewing display in practice, 
and are input from a predetermined user interface to 
express the forward speed (or backward speed), field 
angle, and the like, and is denoted by 110 in Fig. 11. 
The stereoscopic viewing parameters in Fig. 2 are those 
generated by the viewer application program 100 such 
as the base line length, convergence angle, and the like, 
and are denoted by 1 1 1 in Fig. 11. 
[0055] In Fig. 1 1 , the user starts the WWW browser 
101 , which acquires VRML data from a WWW site 200. 
Three-dimensional data described by VRML is 
appended a predetermined file extension. 
[0056] On the other hand, the viewer application pro- 
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gram 100 is registered in advance as a helper applica- 
tion program of the WWW browser 101. More 
specifically, the viewer application program 100 assigns 
an initial value b 0 of the base line length and an initial 
value e 0 of the convergence angle to three-dimensional $ 
space data with the predetermined file extension 
passed from the WWW browser 101, and provides a 
user interface environment for changing space opera- 
tion data including the base line length b, convergence 
angle 6, and the like. w 
[0057] The following explanation will be given with the 
aid of Figs. 11 and 12. Fig. 12 shows the control 
sequence of the viewer application program 100. 
[0058] When the WWW browser 101 detects that data 
received from the WWW site 200 has a file extension, it is 
passes that VRML data to the viewer application pro- 
gram 1 00. The viewer application program 1 00 receives 
the VRML data in step S2. 

[0059] In step S4, initial values (b 0 and G 0 ) of the ster- 
eoscopic viewing parameters (base line length b and 20 
convergence angle 6) are determined from the received 
VRML data. This determination scheme will be 
explained later. The determined initial values are stored 
in a stereoscopic viewing parameter storage area 111. 
[0060] In step S6, the right and left eye images 105 25 
and 104 are generated using equation (1) above in 
accordance with the determined initial values (b 0 and 
G 0 ) of the base line length and convergence angle, and 
are passed to the WWW browser 101. The WWW 
browser 101 or operating system 102 synthesizes these 30 
images and displays the synthesized image on the dis- 
play 10 (step S8). This synthesis scheme uses the sten- 
cil buffer 12, and is one of the characteristic features of 
the viewer apparatus 1000 of this embodiment, as will 
be described in detail later. 35 
[0061 ] The user who observes the display screen may 
change the base line length, field angle, or convergence 
angle, or may make walkthrough operation such as for- 
ward or backward movement, rotation, or the like. The 
operation for changing the base line length, or conver- 40 
gence angle is called stereoscopic viewing parameter 
changing operation in this embodiment. On the other 
hand, the walkthrough operation such as changing in 
field angle, forward or backward movement, rotation, or 
the like is called interaction operation. 45 
[0062] If it is determined that data input in step S10 
indicates end of operation (step S12), the viewer appli- 
cation program 100 ends. 

[0063] On the other hand, if it is determined that the 
input data indicates an interaction operation such as so 
walkthrough, movement of an object in the virtual world, 
or the like, the flow advances to step S22 to execute 
processing corresponding to that interaction operation. 
[0064] Also, if it is determined that the input operation 
is a stereoscopic viewing parameter operation, a stere- ss 
oscopic viewing parameter associated with the opera- 
tion is changed, i.e., the contents in the storage area 
1 1 1 are changed in step S20. 



( Initial Value of Base Line Length ) 

[0065] Of the processes executed in step S4, the initial 
value b 0 of the base line length is determined as follows. 
[0066] More specifically, the viewer application pro- 
gram estimates the initial value b 0 of the base line 
length from only target space data. Note that the 
processing itself for obtaining stereoscopic viewing 
parameters from only space data which is not assigned 
any stereoscopic viewing parameters is not done con- 
ventionally. 

[0067] The size of the entire space is estimated, and 
the product of this estimated size and an appropriate 
constant k is calculated to obtain the base line length 
initial value b 0 . Such determination process means that 
"a size obtained by multiplying the size of the entire 
space by a given constant larger than 0 is defined as the 
size of a human" (the base line length of a standard per- 
son is 65 mm). 

[0068] More specifically, let (X j( Y i( Zj) be a point group 
of VRML data. Among these points (X jp Yj, Zj), the max- 
imum values (X max , Y max , Z max ) and minimum values 
( x min. Y min , 2 min) of Xj, Y it and Z s are obtained. These 
maximum values (X max , Y max> Z^*) and minimum val- 
ues (X min , Y mln , Z min ) give one solid, as shown in Fig. 

13. This solid defines the range of the target space 
expressed by the VRML data. Sizes S in the X-, Y-, and 
2-directions of this solid are respectively given by: 

S x = X max "^min (2) 

^Y =Y max" Y min 

^ Z~*- max *- min 

[0069] Hence, the base line length initial value b 0 is 
given by the maximum values of S x , S Y , and S z , i.e.: 

b 0 =k • max(S x , S r S 2 ) (3) 

where k is a predetermined constant 
[0070] With this method, the user can perceive the tar- 
get space expressed by the VRML data with a given 
size independently of the size expressed by the target 
space. 

[0071] Since the size of the entire space does not 
always serve as a reference for the size of a human, 
desired stereoscopic sense may not always be obtained 
by the base line length determined as described above. 
For example, in order to display an object (e.g., pyramid) 
having a size with respect to a person, as shown in Fig. 

14, the above-mentioned method presents it as an 
object having a size shown in Fig. 15 to the user when 
the size of a human is fixed at a given value, and the 
presented size does not always match the user's 
requirement. However, the base line length obtained by 
the above-mentioned method gives a field of view in 
which the entire target space can be seen. By contrast, 
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it is possible for the present invention to set the base 
line length initial value b 0 at the size of a human eye, but 
such setup is often not preferable. 
[0072] Since the constant value k can be arbitrarily set 
in equation (3) above, the scheme of the embodiment 5 
based on equation 93) allows to set the base line length 
initial value b 0 at the size of a human eye. 

( Determination of Initial Value of Convergence Angle > 

w 

[0073] Of the processes executed in step S4, the initial 
value e 0 of the convergence angle is determined as fol- 
lows. 

[0074] The convergence angle 8 is produced when 
both eyes point inwardly upon observing a close object, is 
The convergence angle can be set when the depth of 
the space is limited and the range in the depth direction 
is narrow. However, under the assumption of observa- 
tion to infinity, it is appropriate to set the convergence 
angle at "0", i.e., to set parallel viewing as an initial 20 
state. Hence, in this embodiment, the initial value of the 
convergence angle is set at: 

e 0 =o 

25 

[0075] The initial value = 0 of the convergence angle 
does not always provide desired stereoscopic sense. 
However, a stereoscopic viewing parameter that can at 
least give stereoscopic sense can be set. 

30 

(User Interface) 

[0076] The initial values of the stereoscopic viewing 
parameters do not give optimal values in every cases, 
as described above. Even when the initial values are 35 
optimal, it may be desirable to change these parameters 
as a result of walkthrough. For example, assume that 
the observer walks toward a given building in the virtual 
space of a street, and approaches the key hole in the 
door knob of one door in that building (then enters that 40 
key hole). In such case, since the right and left images 
cease to merge from a given timing (the deviation 
between the right and left images becomes too large), 
stereoscopic sense is lost. For this reason, the stereo- 
scopic viewing parameters must be able to dynamically 45 
change in correspondence with observer's operation. In 
the aforementioned example, the base line length must 
be reduced (to reduce the size of a human to allow the 
observer to enter the key hole). 

[0077] Furthermore, not only the stereoscopic viewing 50 
parameters but also interaction processing as a result of 
movement of the user in the virtual space is required. 
[0078] The viewer application program 100 also pro- 
vides a user interface for changing the parameters and 
inputting interactions. , 55 
[0079] Fig. 1 6 showsa user interface (GUI) displayed 
on the screen of the display 1 0. 
[0080] Referring to Fig. 16, reference numerals 200 
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and 201 denote user interface tools (BASE LINE 
LENGTH) for changing the base line length b, both of 
which are implemented by slide bars. The slide bar 200 
is used for increasing the base line length, and the slide 
bar 201 is used for decreasing the base line length. 
When the base line length initial value b 0 is set to obtain 
the virtual space shown in Fig. 14, a display shown in 
Fig. 15 is obtained by increasing the base line length 
using the slide bar 200. 

[0081] Referring to Fig. 16, slide bars 202, 203 are 
used for changing the forward speed and backward 
speed (SPEED), respectively. The slide bar 202 is used 
for effecting a large speed change, while the bar 203 is 
for effecting a small change. 

[0082] A slide bar 204 is used for changing the con- 
vergence angle (View Convergence). Furthermore, a 
slide bar 205 is used for changing the field angle (Field 
of View). 

[0083] The user interfaces shown in Fig. 1 are oper- 
ated by the user using the mouse 15. 
[0084] Note that the detailed tools of the user interface 
are not limited to slide bars. For example, tools that can 
continuously change parameters, e.g., graphical jog 
dials, and graphical wheels may be used. 

(Image Synthesis) 

[0085] Synthesis of the right and left images done in 
step S6 in Fig. 12 will be explained below. 
[0086] The right and left eye images to be displayed 
on the display 10 to obtain stereoscopic viewing must 
be stored in a frame buffer. Since the stereoscopic dis- 
play 10 used in this embodiment uses a rear-cross len- 
ticular spectacle-less 3D display, pairs of right and left 
stereo images for stereoscopic viewing are alternately 
displayed in horizontal stripe patterns, as shown in Fig. 
4. For this purpose, one frame buffer must alternately 
store one stripe for the left eye image and one stripe for 
the right eye image, as shown in Fig. 17. 
[0087] Conventionally, as has been described previ- 
ously with reference to Figs. 18 and 19, since a plurality 
of frame buffers are required, a cost reduction cannot be 
attained, and a data read from the frame buffers lowers 
the processing efficiency. 

[0088] However, the buffer control of this embodiment 
requires only one frame buffer, and can obviate the 
need for reading out data from the frame buffer. For this 
purpose, the viewer apparatus 1000 of this embodiment 
comprises the frame buffer 1 1 and stencil buffer 12. 
[0089] The stencil buffer 1 2 is a buffer memory which 
is equipped as an option in some image processing 
apparatuses, in this embodiment, the stencil buffer 12, 
which is a part of a frame buffer, is a memory having a 
1-bit depth, and the same numbers of bits as those of 
the display 1 0 in the horizontal and vertical directions. In 
this embodiment, the stencil buffer 12 is used as a 
mask, as shown in Fig. 20. More specifically, bits "1" 
serve as a mask, and bits "0" do not serve as a mask. 
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Thus, the stencil buffer functions as a mask that masks 
a horizontal line every two lines. 
[0090] Fig. 21 is a flow chart showing the data write 
sequence in the frame buffer, 
f [0091] In step S70, an image obtained when the three- 

dimensional space is viewed by the right eye is gener- 
ated. Then, three-dimensional space data is rendered 
on the basis of the viewpoint position and direction of 
the right eye. In this manner, the right eye image as a 
rendering result is stored in the frame buffer 11. More 
specifically, at this time, data of the right eye image is 
written at all the addresses of the frame buffer 1 1 . 
[0092] In step S72, a mask pattern shown in Fig. 20 is 
prepared, and is written in the stencil buffer 12. Instead 
of using a masking pattern for each image as set forth 
above, lines may be drawn into the stencil buffer so that 
they form a masking pattern as shown in Fig. 20. 
[0093] In step S74, a left eye image is generated, and 
is over-written in the frame buffer. With the processing in 
step S74, the stencil buffer 12 functions so that the right 
eye image remains stored at those pixel positions in the 
frame buffer 1 1 which correspond to "0" positions in the 
stencil buffer 12, and the left eye image is written at 
those pixel positions in the frame buffer 1 1 which corre- 
spond to "1" positions in the stencil buffer 12. With this 
operation, stripes shown in Fig. 17 are obtained. 
[0094] Data transfer that requires processing for read- 
ing out data from a frame buffer, and writing the data in 
another frame buffer is time-consuming processing, and 
lowers the processing speed. However, the scheme 
proposed in Fig. 21 synthesizes the right and left 
images by only overwriting masked data on the single 
frame buffer to update mapped data, without reading 
out data from the frame buffer, thus suppressing any 
processing speed drop. Since only one frame buffer is 
required like in norma! processing for generating an 
image from a three-dimensional space, a cost reduction 
can be achieved. 

( Advantages of Embodiment ) 

[0095] According to the stereoscopic image display 
apparatus described above, the following effects are 
expected. 

i) For example, the base line length, convergence 
angle, and the like are mandatory in stereoscopic 
viewing but are not normally prepared in three- 
dimensional space data. However, according to the 
viewer application program of this embodiment, 
when three-dimensional space data is acquired 
from an external source (especially via the Internet 
or the like), such stereoscopic viewing parameters 
can be automatically added without the intervention 
of the user. The automatic addition of the stereo- 
scopic viewing parameters is particularly effective 
when the viewer application program of this embod- 
iment is implemented as a helper application pro- 



gram of such browsers as cannot limit objective 
data and therefore have to handle various data. In 
this case, the viewer application program can deter- 
mine based on a file extension of data whether or 
5 not three-dimensional space data requires addition 
of parameters. Hence, parameters can be automat- 
ically added, and such determination can be made 
reliably. 

ii) Since the user interface (GUI) that can freely 
10 change the set stereoscopic viewing parameters in 

correspondence with the user's purpose is pre- 
pared, natural stereoscopic viewing can be 
achieved in accordance with the application pur- 
pose. 

15 Since the user interface is a graphical user 

interface (GUI), it has high operability. 

iii) According to the buffer control of this embodi- 
ment, the number of frame buffers can be reduced 
to one since only a low-cost stencil buffer is used to 

20 render a stereoscopic image. Since the need for 
reading out data from the frame buffer can be obvi- 
ated, effective mapping onto the buffer can be 
done. 

25 < Modifications > 

[0096] The viewer application program of the present 
invention is not limited to the helper application program 
of the WWW browsers. It may be implemented as a 
30 plug-in program in the browsers, and it can also serve 
as a standalone program as long as a stereoscopic 
image is to be displayed. 

[0097] The initial values of the stereoscopic viewing 
parameters can also be determined by schemes other 

35 than the aforementioned schemes. Whether or not the 
determined initial values of the parameters make the 
user feel incoherent depends on applications. Hence, 
various constant values k for determining initial values 
may be prepared in advance in the viewer application 

40 program in units of application programs, and different 
constant values may be used in correspondence with 
application programs. 

[0098] The user interface of the present invention 
need not always be displayed on the display, and any 
45 other user interfaces may be used as long as the user 
can change parameters and the like continuously or dis- 
cretely. 

[0099] The mask functions assigned to bits "0" and "1 " 
of the mask information may be reversed. Also, bits "0" 
so and "1" may be stored in correspondence with the lay- 
out order of the image format in the LCD 20 of the ster- 
eoscopic display 10. 

[0100] In the control sequence shown in Fig. 21, the 
left eye image may be written first. Whether or not mask 
55 functions assigned to bits "0" and "1 " of the mask infor- 
mation are reversed may be determined in correspond- 
ence with the layout of the LCD 20 of the stereoscopic 
display 10, 
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[0101] To restate, according to the present invention, 
parameters can be assigned to three-dimensional 
space data in which no stereoscopic viewing parame- 
ters are defined. 
* [0102] Also, the user interface apparatus that can 

freely change stereoscopic viewing parameters can be 
provided. 

[01 03] Furthermore, buffer control for efficient stereo- 
scopic viewing display can be realized using only one 
frame buffer. 

[0104] The present invention can be implemented by 
a computer program operating on a standard desk top 
computer. An aspect of the present invention thus pro- 
vides a storage medium storing processor implementa- 
ble instructions for controlling a processor to carry out 
the method as hereinabove described. 
[0105] Further, the computer program can be 
obtained in electronic form for example by downloading 
the code over a network such as the internet. Thus in 
accordance with another aspect of the present invention 
there is provided an electrical signal carrying processor 
tmplementable instructions for controlling a processor to 
carry out the method as hereinbefore described. 
[0106] As many apparently widely different embodi- 
ments of the present invention can be made without 
departing from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the spe- 
cific embodiments thereof except as defined in the 
appended claims. 

Claims 

1 . A stereoscopic image processing apparatus having 
means for receiving and storing three-dimensional 
space data, characterized by comprising: 

means for generating a parameter required for 
stereoscopic viewing on the basis of the 
received three-dimensional space data. 

2. The apparatus according to claim 1, wherein the 
parameter defines a base line length of a user. 

3. The apparatus according to claim 1 or 2, wherein 
the parameter defines a convergence angle of a 
user. 

4. The apparatus according to claim 1 , further com- 
prising means for determining necessity of genera- 
tion of the parameter on the basis of a file extension 
of the received three-dimensional space data. 

5. A user interface apparatus used upon displaying a 
stereoscopic image, characterized by comprising: 

user setting means for allowing a user to set a 
value of a stereoscopic viewing parameter; and 
means for changing a value of the stereoscopic 
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viewing parameter in accordance with the set 
value. 

6. The apparatus according to claim 6, further com- 
5 prising display means, and wherein said user set- 
ting means displays a GUI on said display means. 

7. The apparatus according to claim 6, wherein said 
GUI is implemented by a slide bar, jog dial or wheel. 

10 

8. The apparatus according to any one of claims 5 to 

7, wherein the stereoscopic viewing parameter 
defines a base line length of the user. 

15 9. The apparatus according to any one of claims 5 to 

8, wherein the stereoscopic viewing parameter 
defines a convergence angle of the user. 

1 0. A stereoscopic image display apparatus for display- 
20 ing a stereoscopic image, characterized by com- 
prising: 

means for allowing a user to set a value of a 
stereoscopic viewing parameter; and 
25 means for changing a value of the stereoscopic 

viewing parameter in accordance with the set 
value. 

11. A buffer control method upon obtaining stereo- 
30 scopic viewing by displaying right and left eye 

images written in one frame buffer, characterized by 
comprising the steps of: 

preparing mask information that defines a 
35 mask function in advance; 

writing one of the right and left eye images in 
the frame buffer; and 

overwriting the other image on the frame buffer 
with reference to the mask information. 

40 

12. A buffer control apparatus which has one frame 
buffer and obtains stereoscopic viewing by display- 
ing right and left eye images written in the frame 
buffer, having a mask memory written with mask 

45 information that defines a mask function in 
advance, characterized by comprising: 

control means for writing one of the right and 
left eye images in the frame buffer, and then 
so overwriting the other image on the frame buffer 

with reference to the mask information in said 
mask memory. 

13. The apparatus according to claim 12, wherein said 
55 mask memory comprises a stencil buffer. 

14. The method according to claim 11 wherein the 
mask function is determined based on an order of 
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stripes on a display apparatus, and a write order of 
images in the frame buffer. 

15. The method according to any one of claims 1 1 and 
14, wherein the mask information is written in a 
stencil buffer at least before the other image is writ- 

• ten in the frame memory. 

16. A program storage medium for a computer program 
storing: 

a step program of receiving and storing three- 
dimensional space data; and 
a step program of generating a parameter 
required for stereoscopic viewing on the basis 
of the received three-dimensional space data. 

17. A program storage medium for a computer program 
storing: 

a step program of receiving and storing three- 
dimensional space data; 
a step program of determining necessity of 
generation of a parameter on the basis of a file 
extension of the received three-dimensional 
space data; and 

a step program of generating a parameter 
required for stereoscopic viewing on the basis 
of the received three-dimensional space data if 
it is determined that the parameter must be 
generated. 

18. A storage medium that stores a buffer control pro- 
gram for obtaining stereoscopic viewing by display- 
ing right and left eye images written in one buffer, 
storing: 

a step program of preparing mask information 

that defines a mask function in advance; 

a step program of writing one of the right and 

left eye images in the frame buffer; and 

a step program of overwriting the other image 

on the frame buffer with reference to the mask 

information. 

19. A viewer apparatus for receiving and displaying 
three-dimensional space data via the Internet, com- 
prising: 

means for storing the three-dimensional space 
data; 

means for generating a parameter required for 
stereoscopic viewing on the basis of the 
received three-dimensional space data; and 
means for rendering the stored three-dimen- 
sional space data on the basis of the parame- 
ter. — 



20. A storage medium storing processor implerhenta- 
ble instructions for controlling a processor to carry 
out the method of any one of claims 11,14 and 1 5. 

5 21. An electrical signal carrying processor implementa- 
ble instructions for controlling a processor to carry 
out the method of any one of claims 11,14 and 1 5. 
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